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ABSTRACT 

Social networks are commonly used for marketing purposes. 
For example, free samples of a product can be given to a 
few influential social network users (or “seed nodes”), with 
the hope that they will convince their friends to buy it. One 
way to formalize marketers’ objective is through influence 
maximization (or IM), whose goal is to find the best seed 
nodes to activate under a fixed budget, so that the number 
of people who get influenced in the end is maximized. Recent 
solutions to IM rely on the influence probability that a user 
influences another one. However, this probability information 
may be unavailable or incomplete. 

In this paper, we study IM in the absence of complete infor¬ 
mation on influence probability. We call this problem Online 
Influence Maximization (OIM) since we learn influence prob¬ 
abilities at the same time we run influence campaigns. To 
solve OIM, we propose a multiple-trial approach, where (1) 
some seed nodes are selected based on existing influence 
information; (2) an influence campaign is started with these 
seed nodes; and (3) users’ feedback is used to update influ¬ 
ence information. We adopt the Explore-Exploit strategy, 
which can select seed nodes using either the current influence 
probability estimation (exploit), or the confidence bound on 
the estimation (explore). Any existing IM algorithm can be 
used in this framework. We also develop an incremental algo¬ 
rithm that can significantly reduce the overhead of handling 
users’ feedback information. Our experiments show that our 
solution is more effective than traditional IM methods on 
the partial information. 

1. INTRODUCTION 

In recent years, there has been a lot of interest about how 
social network users can affect or influence others (via the 
so-called word-of-mouth effect). This phenomenon has been 
found to be useful for marketing purposes. Eor example, 
many companies have advertised their products or brands 
on social networks by launching influence campaigns, giving 
free products to a few influential individuals (seed nodes), 
with the hope that they can promote the products to their 
friends [19| . The objective is to identify a set of most influ¬ 
ential people, in order to attain the best marketing effect. 

‘Work mainly done while the author was affiliated with 
University of Hong Kong. 



Figure 1: The OIM framework. 


This problem of influence maximization (IM) has attracted 
a lot of research interest 

Given a promotion budget, the goal of IM is to select the 
best seed nodes from an influence graph. An influence graph 
is essentially a graph with influence probabilities among nodes 
representing social network users. In the independent cascade 
model, for example, a graph edge e from user a to 6 with 
influence probability p implies that a has a chance p to 
affect the behavior of b (e.g., a convinces b to buy a movie 
ticket) [16| . Given an influence graph, IM aims to find k 
seed nodes, whose expected number of influenced nodes, or 
influence spread, is maximized. Marketing efforts can then 
be focused on the k nodes (or persons). In the IM literature, 
these seed nodes are said to be activated ilziraiH- 

While existing IM algorithms effectively obtain the most 
influential seed nodes, they assume that the influence proba¬ 
bility value between each pair of nodes is known. However, 
this assumption may not hold. Consider a marketing firm 
starting in a new city with some knowledge of the social 
network of the users in the city. The company, however, 
does not know how influence propagates among these users. 
Unless the influence probability information is known, the 
marketing firm cannot run an IM algorithm and decide the 
target users. To obtain these values, action logs, which record 
the social network user’s past activities, can be used [12| . 
This information may not be readily available. 

Is it possible to perform IM on a social network, even if the 
information about influence probabilities is absent or incom¬ 
plete! We call this problem Online Influence Maximization 
(OIM), as we aim at discovering influence probabilities at 
the same time we are performing influence campaigns. (We 
say that an IM algorithm is offline, if it assumes that the 









































influence probability between every node pair is known in 
advance.) In the absence of complete influence probability 
information, making the best marketing effort out of a lim¬ 
ited promotion budget can be challenging. To tackle this 
problem, we propose a solution based on influencing seed 
nodes in multiple rounds. In each round, we select some 
seed nodes to activate (e.g., advertising a product to a few 
selected users). The feedback of these users is then used 
to decide the seed nodes to be activated in the next round. 
The information about influence probabilities in the social 
network is learnt and refined during these campaigns. 

Figure [^illustrates our OIM framework. It contains multi¬ 
ple successive influence campaigns, or trials. A trial should 
fulfill one of two objectives: (1) to advertise to promising 
nodes; and (2) to improve the knowledge about influence 
probabilities. A trial consists of two phases: selection and 
action. In the selection phase, an uncertain influence graph 
is maintained. This graph models the uncertainty of influ¬ 
ence probabilities among social network users, in terms of 
a probability distribution. A seed selection strategy, based 
on an existing IM solution, is then executed on this graph 
to produce up to k seed nodes. In the action phase, the 
selected seed nodes are activated in the real world (e.g., send¬ 
ing the advertisement message to chosen users). The actions 
of these users, or feedback (e.g., whether the message is fur¬ 
ther spread), is then used to update the uncertain influence 
graph. The iteration goes on, until the marketing budget is 
exhausted. In this paper, we focus on the two crucial compo¬ 
nents of the selection phase: (1) seed selection strategy, and 
(2) techniques for updating the uncertain influence graph. 

1. Seed selection strategy. To choose seed nodes in a 
trial, a simple way is to make use of existing IM algorithms. 
Due to the lack of knowledge about influence probabilities, 
this approach may not be the best. We thus develop an 
Explore-Exploit strategy (or EE), which performs IM based 
on existing influence probability information: 

• [Exploit] Select k seed nodes for getting the most reward¬ 
ing influence spread from the influence graph, derived from 
the uncertain influence graph. Any state-of-the-art IM algo¬ 
rithms (e.g., CELF [18[ , DD |^, TIM and TIM-1- [26| ) can 
be used; or 

• [Explore] Select k seed nodes based on some strategy (e.g., 
through estimating the confidence bound of the influence 
probability) to improve the knowledge about the influence 
graph. 

In this paper, we study strategies for exploit and explore. 
With suitable use of strategies, EE performs better than 
running an existing IM algorithm on the uncertain influence 
graph alone. 

In our OIM solution, N trials are carried out. In each trial, 
an existing IM algorithm may be executed. If N is large, the 
performance of our algorithm can be affected. The problem 
is aggravated if the underlying uncertain influence graph is 
big. For state-of-the-art IM algorithms (e.g., CELF [^ and 
TIM-1- [26| ), this running time is dominated by the cost of 
sampling the influence graph. For example, in TIM-f, the 
sampling effort costs more than 99% of the computation 
time. We design an efficient solution, based on the intuition 
that users’ feedback often only affects a small portion of the 
influence graph. If samples of the previous iterations are 
stored, it is possible to reuse them, instead of sampling the 
influence graph again. We examine conditions allowing a 


sampled graph to be effectively reused in a new trial. We 
propose an incremental algorithm, and present related data 
structures for facilitating efficient evaluation of our solution. 
This algorithm can support any sample-based IM algorithm 
running on independent cascade models. We demonstrate 
how to use TIM-f in this paper. 

2. Updating the uncertain influence graph. As dis¬ 
cussed before, our seed selection strategy is executed on the 
uncertain influence graph (Figure [^. It is important that 
this graph accurately reflects the current knowledge about 
the influence among different users, so that the seed selec¬ 
tion strategy can make the best decision. We investigate 
algorithms for updating this graph based on the feedback of 
activated users (e.g., whether they spread out an advertise¬ 
ment message). We examine two variants, which update the 
influence graph locally and globally. A local update refreshes 
the parameters of the influence probability distribution be¬ 
tween two graph nodes, while a global update is applied 
to the parameters of the influence probability information 
that applies to the whole uncertain influence graph. These 
algorithms are developed based on classical machine learn¬ 
ing methods (e.g, Least Squares Estimation and Maximum 
Likelihood). 

Our solutions can be adopted by social marketers who 
aim to promote their products, in cases when the underly¬ 
ing probabilities of the influence graph are unknown. Our 
approach can utilize any state-of-the-art IM algorithm. We 
also examine how to update the uncertain influence graph 
effectively by machine learning methods. We develop an in¬ 
cremental algorithm to improve the efficiency of our solution. 
Our experiments demonstrate that our proposed methods 
can effectively and efficiently maximize influence spread. 


2. RELATED WORK 

Influence Maximization (IM). Kempe et al. 16 first 
proposed the study of IM in social networks. They showed 
that finding the set of seed nodes that maximizes influence 
is NP-hard, and showed that the greedy algorithm has a con¬ 
stant approximation guarantee. However, this solution is not 
very fast, because thousands of samples are often required, 
and each sampling operation has a complexity linear to the 
graph size. To improve the efficiency of IM solutions, sev¬ 
eral heuristics were developed, namely Degree Discount [^, 
PMIA 1^, IPA [^, and IRIE [^. Although these heuristics 
are fast, their accuracy is not theoretically guaranteed. Im¬ 
proved aproximation algorithms with theoretical guarantees 
include CELF [^, CELF-f-f [^, and NewGreedy [^. More 
recently, Borgs et al. proposed an algorithm based on reverse 
influence sampling, and showed that it is runtime-optimal 
with accuracy guarantees [^. The scalability of this solu¬ 
tion was enhanced by Tang et ah, who developed TIM and 
TIM-1- 26 to further reduce the number of samples needed. 

There are also other works that address different variants 
of the IM problem: (1) incorporating community [27] and 
topic [^ information in the propagation process; (2) compe¬ 
tition of different parties for influence [20[ ; and (3) use of 
other influence propagation models such as linear threshold 
or credit distribution [11||14|[^ . 

Learning influence probabilities. Saito et al. [^ mod¬ 
eled the problem of obtaining influence probabilities as an 
instance of likelihood maximization, and developed an ex¬ 
pectation maximization algorithm to solve it. Given a social 
network and an action log (e.g., user u performs action a at 




Table 1: Symbols used in this paper. 


symbol 

description 

G 

influence graph 

V 

set of users (nodes) of G 

E 

set of edges of G 

Pij 

influence probability from i to j (fixed value) 

P ■ 

influence probability from i to j (random variable) 

N 

number of trials 

k 

budget for each trial 

s 

set of seed nodes 

cr(S) 

expected influence spread 

(«./3) 

global prior for the beta distribution 


set of successfully activated nodes in trial n 

F„ 

real world activation feedback in trial n 

ihij,mij) 

number of successful and unsuccessful 
activations of the edge from i to j 


time t), Goyal et al. developed static and time-dependent 
models to compute influence probabilities between a pair of 
social network users. These methods require the action log 
information of all the users involved to be known in advance; 
however, this information may not be available. Our frame¬ 
work does not require all action logs to be available. Instead, 
we select seed nodes in multiple advertising campaigns, so 
that influence maximization can be done faster. We then use 
users’ feedback in each campaign to learn and rehne influence 
probabilities. 

Multi-armed bandits (MAB). The EE strategy in the 
seed selection phase of our solution is inspired by the e- 
greedy algorithm, which was originally developed to solve 
the multi-armed bandit problem (MAB) [^. In the e-greedy 
algorithm [^, e controls the trade-off between exploitation 
and exploration. Specifically, with probability 1 - e, an action 
is executed based on the current knowledge (i.e., exploit)- 
with probability e, another action is performed (i.e., explore). 
This framework is adopted as a baseline in our solution. 

studies combinatorial MAB algorithms, and in par¬ 
ticular the CUCB algorithm, which uses upper confidence 
bounds for choosing between explore and exploit. A sce¬ 
nario akin to the OIM problem is illustrated and it is shown 
that CUCB achieves a bound on the regret. However, CUCB is 
not applicable due to two factors. First, the activated nodes 
are counted multiple times leading to redundant activations 
and choices. Second, and most practically important, the 
approximation bound depends on an initialization step in 
which each arm (in this scenario, seed node) is tested to get 
an activation feedback; this is not practically feasible in cases 
when activation budgets are limited. Another algorithm 
closely related to our framework is Thompson Sampling 
where each independent arm is simulated by a Beta distri¬ 
bution of successes and failures. In our scenario, the arms 
are the parameters of the algorithms, and defining success 
and failure in a result of an influence maximization is not 
trivial. 

3. INFLUENCE MAXIMIZATION: REVIEW 

We now provide a review of the IM problem and its solu¬ 
tions. This forms the basis of the OIM problem to be studied 
in this paper. Tableshows the notation used. 

Let G = {V, E,p) be an influence graph, where v £ V are 
users or nodes, and e £ E are the links or edges between them. 
Each edge e = {i,j) between users i and j is associated with 
an influence probability pij £ [0,1]. This value represents the 


probability that user j is activated by user i at time t 1, 
given that user i is activated at time t. We also suppose that 
time flows in discrete, equal steps. In the IM literature, pij 
is given for every i and j. Obtaining pij requires the use of 
action logs [12| which may not be available. In this paper, 
we investigate how to perform IM without knowing pij in 
advance. 

In the independent cascade model, at a given timestamp t, 
every node is in either active (influenced) or inactive state, 
and the state of each node can be changed from inactive to 
active, but not vice-versa. When a node i becomes active in 
step t, the influence is independently propagated at t-l-1 from 
node i to its currently inactive neighbors with probability pij. 
Node i is given one chance to activate its inactive neighbor. 
The process terminates when no more activations are possible. 
A node can be independently activated by any of its (active) 
incoming neighbors. Suppose that the activation process 
started from a set S of nodes. We call the expected number 
of activated nodes of S the expected influence spread, denoted 
o-{S). Formally: 

Definition 1. Given a weighted graph G = {V,E,p), let 
infi be the immediate influence operator, which is the random 
process that extends a set of nodes X <£ V into a set of 
immediately influenced nodes infl(X), as follows: 

fl ifv£X; 

Pr(u G mfl(X)) - ^ otherwise, 

y uex 

Given a seed set S C V, we define the set of influenced nodes 
I{S) £- V as the random variable that is the fixpoint I°°{S) 
of the following inflationary random process: 

I°iS) = 0; 

< f{S) = S; 

^ r+'^{S) = r+\S) U infl(7"+^(S')\7"(S')) for n > 0. 
The influence spread o-{S) is £[17(5)|]. 

Based on the above definition, |16| defines the influence 
maximization problem (IM) as follows. 

Problem 1. Given a weighted graph G = {V,E,p) and 
a number 1 ^ fc ^ |1/|, the influence maximization (IM) 
problem finds a set S C V such that (j{S) is maximal subject 
to ISI = k. 

As discussed in [^, evaluating the influence spread is difficult. 
Even when the spread values are known, obtaining an exact 
solution for the IM problem is computationally intractable. 
Next we outline the existing IM algorithms for this problem. 

IM algorithms. A typical IM algorithm evaluates the 
score of a node based on some metric, and inserts the k best 
nodes, which have the highest scores, into S. For example, 
the degree discount (DD) heuristic selects the nodes with 
highest degree as S. Another classical example is greedy. 
at each step, the next best node, or the one that provides 
the largest marginal increase for a, is inserted into S. This 
is repeated until \S\ = k. The greedy algorithm provides 
an (1 — l/e)-approximate solution for the IM problem. To 
compute the influence spread efficiently, sampling-based al¬ 
gorithms with theoretical guarantees were developed. For 
example, CELF evaluates the expected spread of nodes 
with the seed nodes, and select the nodes with the largest 
marginal spread; TIM counts the frequencies of the nodes 






appearing in the reversed reachable sets, and chooses the 
nodes with the highest frequencies; TIM+ 26 is an extension 
of TIM for large influence graphs. 

We say that the above IM algorithms are ojfline, since 
they are executed on the influence graph once, assuming 
knowledge of Pij for every i and j. If these values are not 
known, these algorithms cannot be executed. This problem 
can be addressed by online IM algorithms, as we will discuss 
next. 


4. MAXIMIZING INFLUENCE ONLINE 

The goal of the online influence maximization (or OIM) 
is to perform IM without knowing influence probabilities in 
advance. Given a number N of advertising campaigns (or 
trials), and an advertising budget of k units per trial, we 
would like to select up to k seed nodes in each trial. These 
chosen nodes are then advertised or activated, and their 
feedback is used to decide the seed nodes in the next trial. 
Let us formulate the OIM problem below. 


Problem 2. Given a weighted graph G = {V,E,p) with 
unknown probabilities Puv, o,nd a budget consisting of N 
trials with 1 ^ fc ^ |P| activated nodes per trial, the online 
influence maximization (OIM) problem is to find for each 
1 ^ n ^ N a set Sn. of nodes, with ^ k, such that 

E [|Ui<n^jv “ maximal. 


Note that the IM problem, discussed in Section is a 
special case of the OIM problem (by setting = 1). Since 
solving the IM problem is computationally difhcult, finding 
a solution for the OIM is also challenging. We propose 
a solution that consists of multiple trials. In each trial, 
a selection (for choosing appropriate seed nodes) and an 
action (for activating the seed nodes chosen) is performed 
(Figure]^. The seed selection makes use of one of the offline 
IM algorithms discussed in Section 


We next present the uncertain influence graph, which cap¬ 
tures the uncertainty of influence probabilities (Section 4.1 1 . 
We then discuss our solution based on this graph in Sec¬ 
tion IT^ 


4.1 The Uncertain Influence Graph 

We assume that a social network, which describes the 
relationships among social network users, is given. However, 
the exact influence probability on each edge is not known. We 
model this by using the uncertain influence graph, in which 
the influence probabilities of each edges are captured by 
probability density functions, or pdf (Figure Q. The pdf can 
be refined based on the feedback returned from a trial. Since 
influence activations are binary random variable, we capture 
the uncertainty over the influence as a Beta distribution. 
Specifically, the random variable of the influence probability 
from node i to node j, Pij is modeled as a Beta distribution 
having probability density function: 


fPij (®) 


B{aij, ftij) 


where B(aij,/3ij) is the Beta function, acting as a nor¬ 
malization constant to ensure that the total probability 
mass is 1, and onj and Pij are the distribution param¬ 
eters. For the Beta distribution, E[Pij] = ^ and 

^In this paper we assume that the advertising budget k is 
fixed for each trial. 


An advantage of using the 
Beta distribution is that it is a conjugate prior for Bernoulli 
distributions, or more generally, binomial distributions. This 
allows us to compute the posterior distributions easily when 
new evidence is provided. Section explains this in more 
detail. 

At the time of the first trial, we assume no prior informa¬ 
tion about the influence graph, except global a and p pa¬ 
rameters, shared by all edges, i.e., Pij ~ B{a,p) V{i,j)€E. 
These global a and P parameters represent our global prior 
belief of the uncertain influence graph. In the absence of any 
better prior, we can set a = p = 1, with B{1, 1) being the 
uniform distribution. 

Our model can be extended to handle various prior infor¬ 
mation about the influence graph. For example, if we have 
individual prior knowledge {aij,pij) about an edge, we can 
set Pij as Pij ~ B{aij, Pij). When we have access to only 
the mean and variance of the influence of an edge, we can 
derive aij and Pij from the formulas of E[Pij] and o-'^[Pij] 
given above. For the situation in which some action logs 
involving the social network users are available, algorithms 
for learning the influence probabilities from these logs 
can be first applied, and the estimated influence probabilities 
can then be used as prior knowledge for the graph. 

4.2 The OIM Framework 


Algorithm 1 Framework(G, k, N) 

1: Input: trials N, budget k, uncertain influence graph 
G 

2: Output: seed nodes Snin — 1... N), activation results 
A 

3: 0 

4: for n = 1 to A do 
5: Sn. t— Choose(G, k) 

6: {An,Fn) ■«— RealWorld(S'n) 

7: A i — A U An 

8: Update(G, FVi) 

9: return {S'„|n = 1... N}, A 


Algorithm depicts the solution framework of the OIM 
problem. In this algorithm, N trials are executed. Each 
trial involves selecting seed nodes, activating them, and 
consolidating feedback from them. In each trial n (where 
n = 1,..., N), the following operations are performed on the 
uncertain influence graph G: 

1. Choose (Line |^: A seed set Sn is chosen from G, 
by using an offline IM algorithm, and strategies for 
handling the uncertainty of G (Section]^. 

2. RealWorld (Lines p]|^: The selected seeds set is tested 
in the real world (^g., sending advertisement messages 
to selected users in the social network). The feedback 
information from these users is then obtained. This is 
a tuple {An,Fn) comprised of: (i) the set of activated 
nodes An, and (ii) the set of edge activation attempts 
En, which is a list of edges having either a successful 
or an unsuccessful activation. 

3. Update (Line|^: We refresh G based on {A„,F„) (Sec¬ 
tion . 

One could also choose not to update G, and instead only 
run an offline IM based on the prior knowledge. Our ex¬ 
perimental results show that the influence spread under our 
OIM framework with proper updates is better than the one 
















without any update. Next, we investigate the design and im¬ 
plementation of Choose (Section]^ and Update (Section]^. 


5. CHOOSING SEEDS 


We now study two approaches for selecting k seed nodes 
in the Choose function of Algorithmheuristic-based (Sec¬ 
tion 5.11 and explore-exploit strategies (Section 5.2 I. 


5.1 Heuristic-Based Strategies 

We first discuss two simple ways for choosing seeds from 
the uncertain influence graph G. 

1. Random. This heuristic, which arbitrarily selects k seed 
nodes, is based on the fairness principle, where every user 
has the same chance to be activated. 

2. MaxDegree. Given a node p in G, we define the out- 
degree of p to be the number of outgoing edges of p with 
non-zero influence probabilities. The out-degree of p can mean 
the number of friends of the social network user represented 
by p, or their number of followers. Intuitively, if p has a 
higher out-degree, it has a higher chance of influencing other 
users. The MaxDegree heuristic simply chooses the nodes 
with k highest out-degree values. 

The main advantage of these two heuristics is that they 
are easy to implement. However, they do not make use 
of influence probability information effectively. In a social 
network, some users might be more influential than others. 
It may thus be better to target users with higher influence 
probabilities on their outgoing edges. The above heuristics 
also do not consider the feedback information received from 
the activated users, which can be useful to obtain the true 
values of the influence probabilities. We will examine a better 
seed-selection method next. 


5.2 Explore-Exploit Strategies 

The Explore-Exploit (EE) strategy chooses seed nodes 
based on influence probabilities. Its main idea is to exploit, 
or execute an offline IM algorithm, based on the influence 
information currently available. Since this information may 
be uncertain, the seed nodes suggested by exploit may not 
be the best ones. We alleviate this problem by using explore 
operations, in order to improve the knowledge about influence 
probabilities. Solutions for effectively controlling explore 
and exploit operations have been studied in the multi-armed 
bandit (MAB) literature [2lp3l . These MAB solutions inspire 
our development of the two seed-selection strategies, namely 
e-greedy and Confidence-Bound (CB). Next, we present these 
two solutions in detail. 


1. £-greedy. In this strategy (Algorithm]^, a parameter 
e is used to control when to explore and when to exploit. 
Specifically, with probability 1 — e, exploitation is carried 
out; otherwise, exploration is performed. 


Algorithm 2 £-greedy(G, k) 

1: Input: uncertain influence graph G = {V,E,P), budget 
k 

2: Output: seed nodes S with IS"! = fc 
3: sample 2 : from Bernoulli{e) 

4: if 2 = 0 then S Explore(G, k) 

5: else S Exploit(G, fe) 

6: return S 


In Exploit, we execute an offline IM algorithm, given the 
graph information we have obtained so far. Recall that we 
model the influence probability pij between nodes i and j 
as a probability distribution Pij. We use the mean of Pij to 
represent pij, i.e., 


Pij — lE[Pij] 


Oiij -f Pij 


A graph with the same node structure but with the Pij values 
on edges constitutes an influence graph G', on which the 
offline IM algorithm is executed. Notice that when e = 0, 
the solution reduces to exploit-only, i.e., the IM algorithm is 
run on G' only. 

The main problem of Exploit is that estimating pij by 
E[Pij] can be erroneous. For example, when Pij is a highly 
uncertain Beta distribution (e.g., the uniform distribution, 
B(l, 1)), any value in [0,1] can be the real influence proba¬ 
bility. Let us consider a node i that has, in reality, a high 
influence probability pij on another node j. Due to the large 
variance in Pij , its value is underestimated. This reduces the 
chance that Exploit chooses node i to activate; consequently, 
the seed nodes selected may not be the best. The Explore 
routine is designed to alleviate this problem. Rather than 
equating Pij to E[Py], pij is over-estimated by using Pij's 
standard deviation, or Oij-. 


Pij 


E[Pi3] -I- aij 

^ (n -i- ! '^*tPij ^ 

CHij Pij \ ^ ^ Oiij + Pij + 1 / 


Then an offline IM algorithm on these new values of Pij is 
performed. A node i that has a small chance to be chosen 
may now have a higher probability to be selected. Our 
experiments show that the use of Explore is especially useful 
during the first few trials of the OIM solution, since the 
influence probability values during that time may not be 
very accurate. From the feedback of activated users, we can 
learn more about the influence probabilities of the edges of i. 
We will discuss this in detail in Section |6l 

This e-greedy algorithm has two problems. First, it is 
difficult to set an appropriate e, which may have a large 
impact on its effectiveness. Second, increasing pij by aij may 
not always be good. Based on these observations, we next 
propose an improved version of e-greedy. 


2. Confidence-Bound (CB). The main idea of this strat¬ 
egy is to use a real-valued parameter 6 to control the value 
of Pij-. 

Pij = E[Pij] + 9aij. (5.1) 

As shown in Algorithmj^ for every edge e from node ito j, we 
compute its mean pij, variance aij, and influence probability 
Pij based on 9 (Lines 3-6). An offline IM algorithm is then run 
on G', the influence graph with the probabilities computed 
by Equation |5.1| (Lines 7-8). The set S of seed nodes is then 
returned (Line 9). 

Setting 9. The key issue of Algorithm is how to deter¬ 
mine the value of 9, so that the best S can be found. Observe 
that when 9 — 0, pij becomes pij or E[Pij], and CB reduces 
to Exploit of the e-greedy algorithm. On the other hand, 
when 9 — 1, pij becomes E[Pij] -|- aij, and CB is essentially 
Explore. Thus, e-greedy is a special case of CB. However, CB 
does not restrict the value of 9 to zero or one. Thus, CB is 
more flexible and general than e-greedy. 















Algorithm 3 CBCG, fc) 

1: Input: uncertain influence graph G = {V,E,P), budget 
k 


2 

3 

4 

5 

6 

7 

8 
9 


Output: seed nodes S with l^l = fc 
for e £ E do 

k-ij ^ E-Tp- 

1 j ^ 

Pij ■<— pij + Ooij 

G' •<— G, with edge probabilities G E 

s^m{G',k) 

return S 



Figure 2: Updating the influence graph and 9 with user 
feedback. 


In general, when 0 > 0 is used, it means that CB consid¬ 
ers the influence probabilities given by pij’s to be under¬ 
estimated, and it attempts to improve the activation effect 
by using larger values of Pij. On the contrary, if 0 < 0, the 
influence probabilities are considered to be over-estimated, 
and CB reduces their values accordingly. As we will discuss 
in Section [63] 9 can be automatically adjusted based on the 
feedback returned by activated users. This is better than 
e-greedy, where the value of e is hard to set. Note that we 
choose to use a global 9 instead of a local one on each edge, 
to reduce the number of parameters to be optimized and to 
improve efficiency. 

6. MANAGING USER FEEDBACK 

Recall from Algorithm that after the seed nodes S are 
obtained from Choose (Line 5), they are activated in the real 
world. We then collect feedback from the users represented by 
these nodes (Lines 6-7). The feedback describes which users 
are influenced, and whether each activation is successful. For 
instances of such feedback traces, take for example Twitter 
and other micro-bloggin platforms. In these, the system can 
track actions such as likes and retweets which are reasonable 
indicators of influence propagation. We now explain how to 
use the feedback information to perform Update (Line 8), 
which refreshes the values of influence probabilities and 9 
used in the CB algorithm. 

Given a trial n in Algorithm[^ let A„ be the set of activated 
nodes in that trial, and Fn. be the set of activation results. 
Specifically, Fn contains tuples in the form of {i,j, aij), where 
i and j are users between which an activation was attempted; 
aij = 1 if the influence was successful, and atj = 0 otherwise. 
Note that {i,j) is an edge of the influence graph G. Also, F„ 
might not contain all edges of G, since an activation might 
not reach every user in G. 

Three kinds of updates can be performed based on An and 


Fn. 

1. Local (Section |6.l[ ): Update the influence probability’s 
distribution (i.e., B(aij, fiij)) if the edge (i.e., activation from 
i to j) was attempted; 

2. Global (Section |6.2[ ): Update the global prior informa¬ 
tion a and /?, which are shared by all edges of G; and 

3. 9 (Section |6.3[ ): Update the value of 9 used in CB, if it 
is used as a seed selection strategy in Choose. 

Figure [^illustrates these three kinds of updates in the n-th 
trial. In the next sections, we discuss how to conduct these 
updates in detail. We remark that these update methods 
do not affect Random and MaxDegree, since they do not use 
these updated values. 

6.1 Updating Local a and /3 

As we mentioned before, the influence probability be¬ 
tween any two adjacent nodes i and j is modeled as a 
Beta distribution with parameters and pij, denoted as 
Pij ~ Yi^aij, Pij). Since the Beta distribution is a conjugate 
prior for the Bernoulli distribution, then, given feedback 
{i,j,aij) in Fn (seen as a Bernoulli trial), we can update the 
distribution as follows: 

1. If aij = 1, i.e., the activation from node i to node j 
was successful: Pij ~ ^{ciij + l,Pij)', 

2. If aij = 0, i.e., the activation from node i to node j 
failed: Pij ~ B{aij,Pij + 1). 

In the beginning, we have no prior information about 
the distribution except the global a and P, i.e., aij = a 
and Pij = p. After n trials and activations, we have thus 
collected n pieces of feedback information. Let hij (niij) be 
the number of successful (failed) activations for edge (*, j). 
We have 


aij — a hij, Pij — P Triij. 

Hence, this local update is equivalent to maintaining a distri¬ 
bution B{a + hij, P + rriij), i.e., the distributions on the edges 
simply count the number of successful and failed activations 
passing through that edge, smoothed by the prior B(a, P). 

Note that this update process corresponds exactly to the 
MLE approach taken by [^ to learn influence probabilities 
from action logs, with a smoothing prior added. The im¬ 
portant difference is that [^ only conducts this estimation 
for edges where there is evidence, i.e., local updates. If the 
evidence is sparse, this can lead to a sub-optimal, and over- 
fitting, influence graph. Global update of Beta priors, which 
go beyond the local feedback, can yield a better influence 
graph. 

6.2 Updating Global d and p 

Local updates to the random variable Pij allows the edge 
influence probability distribution to be updated directly. In 
the first few trials, however, the real influence spread is sparse 
and limited, and most of the edges will not be reached by 
an activation. Therefore, the influence of choosing a good 
prior will weigh heavily on how Choose performs. Once some 
evidence is gathered, this prior can be refined by taking into 
account the feedback in a global sense, over all trials up to 
the current one. Next, we present two methods of updating 
the global a and P priors based on the feedback. 

Least Squares Estimation. The first solution is to find the 
best fit for the a and P priors according to the real spread 
that we obtained from the real world test at each trial. 



























Let us first explain the reasoning when there is one seed 
node (i.e., I^nl = 1), and we fix a = 1. Let An be the 
set of successful activated nodes before the n-th trial (i.e., 
An ~ and fr„({i}) be the expected number of 

additional activated nodes (or expected additional spread) 
from the seed node i in the n-th trial. For S„ = {s}, u'„({s}) 
is: 


Maximum Likelihood Estimation. Given the feedback from 
each trial n, we can compute the likelihood of the feedback 
Fn given the probabilities of each edge in the feedback tu¬ 
ples, by assuming they are activated independently. The 
likelihood depends on the successful activations (hits) and 
failed activations (misses) of each edges and the global prior 
parameters a and /3: 


- 4 M) = i+ E Psi X (Jn{{i}) + E ^ ('^«({*}) - 1 )> 

(s,i)gB (s,i)eE 

which is the sum of the outgoing spreads weighted by the 
outgoing probabilities Pai and discounted by 1 for nodes 
already activated along an outgoing edge. 

We estimate cr„({s}) by \An\ from the feedback obtained by 
the influence campaign. We also estimate pai = , 

i.e., the mean of dij). Note that hai -f ruai is the total 

number of attempts from node s to i, which is the same 
for neighbors of s because every activation through s tries 
to activate all outgoing nodes in the independent cascade 
model. Thus, we use ta to denote hai -b rriai Wis,i)€E. By 
further estimating cr„({i}) by an overall estimation an and 
set a = 1 , we obtain 

\An\ = 1+ a I , I 1 ( E {hai + l)an— E (hai + 1) 

P \(s.i)eB (a,i)eE,ieAn 

Let Oa be the outgoing degree of s, as be the number of 
(previously) activated neighbors of s (i.e., as = |{i|(s,i) £ 

EAi £ ^n}|), ha be the number of total successful activations 
(or hits) on outgoing edges of s, and haa be the number of 
total hits on edges leading to activated neighbors. The above 
equation is simplified to 

(|^n| — l)/3 = (1 — |^n|)(ts -f 1) -f {ha F Os)dn — {has -f Ug). 

We then rewrite it as the form of 2;„/3 = j/„. Since this 
equation also applies to activations in all trials up to the 
current one, we use the least square estimator for linear 
regression without an intercept term to get an estimator for 

i3 = {F-y) / {x-x), 

where x and y are the vectors of values Xn and j/n. The same 
principles apply when estimating a and j3 simultaneously, 
and we omit the details here. 

We estimate an by the average spread of the node from 
the activation campaigns, i.e., (t„ = 

Note that, when (t„ = 0, the equation for \An\ is exactly 
the degree discount estimator from the IM literature [^, and 
represents a lower bound on the spread from a node. 

A further complication occurs when > 1, which might 
result in an equation at least quadratic in /3, due to the 
influence probability equations of nodes which are neighbors 
of more than one seed node. In this work, we simplify the 
estimation by assuming full independence among seed nodes, 
and hence replacing Xn and pn by the sum over all s £ 5'„. 

We remark that the estimator above suffers from the re¬ 
liance on the spread estimation (t„. However, it is a good 
option when we cannot access the full activation feedback 
Fn, but instead, do have the access to the set of successful 
activated nodes in each trial (i.e., the set An). This may 
happen in an alternate problem setting when one cannot get 
all the feedback information from the activated users in An. 


C{Fn) 


L{Fn I a,f)) 


n pT/{'^-Piif “'L 

a + P + hij+rriij 


We need to find the parameters a and P which maximize 
the likelihood: 


argmax/l(F'„ | a,P). 

To simplify calculations we take the maximum of the log 
likelihood: 


log£(F’„ \ a,P) = E log(“ + 

(i,j,aij)£En 

+ E {i-aij)log{p + mij) 

(i,3,aij)£Pn 

- E log{a + P + hij + rriij). 

(i,3,aij)eFn 


The optimal values are obtained by solving the equations 
ai°g£(F„|o,/ 3 ) ^ Q aiosC(Fn\c,p) ^ 0 for a and P, respec¬ 
tively, which can be simplified as 


E 

(t,j,aij)£Fn.,aij = l 


1 

a -I- hij 


E 

(i,j,aij)£Fn,aij-0 


1 

P + rriij 


This equation can be solved numerically by setting a and 
solving p. In practice, we can fix a = 1 , and let f{P) be 


/(/^) = E 

(hi.Oi,)eF„ 


P + rrii 


E 


aFhi 


Since /(/3) is a monotonically decreasing function of P {f'{P) F, 
0 V/? Is 1), we can solve f{P) = 0 by a binary search algo¬ 
rithm with an error bound 77 (e.g., 10“®). And thus, the 
global a and P priors are updated accordingly. 


6.3 Updating B 

We now explain how to dynamically update the value of B 
used in the CB strategy (Section |5.2[ ). 

Let B = {B\,B 2 , . ■. ,Bq\ be the q possible values of 6. We 
also let 0 = ■ ■ ■ jFq}: where ipj is the probability of 

using Bj in CB. Initially, ipj = 1/q for j = 1,..., g, and its 
value is updated based on the gain obtained in each trial. 
The gain is defined as Gn = |A„|/|H|, where |A„| is the real 
influence spread observed in each round. We then determine 
8 by using the exponentiated gradient algorithm [^. The 
rationale of using this solution is that if the value of Bj used 
in this trial results in a high gain, the corresponding ipj will 
be increased by the algorithm, making Bj more likely to be 
chosen in the next trial. Algorithm gives the details. 

Here, 7 and A are smoothing factors used to update weights, 
and I[z] is the indicator function. We compute 0 by normal¬ 
izing vector w with regularization factor r. All the values in 
w are initialized with the value of 1 . 











Algorithm 4 ExponentiatedGradient(ip, <5, G„, j, w) 


1 : 

2 : 

3: 

4: 

5: 

6 : 

7: 

8 : 


Input: (p, probability distribution; 5, accuracy parame¬ 
ter; Gn, the gain obtained; j, the index of latest used 6j-, 
w, a vector of weights; N, the number of trials. 

Output: 9 


for i = 1 to 5 do 


497 \ , _2L 

3-1-7 ’ ^ ^ 29 


Wi^WiX exp (a X G"X^h-t]+7 j 


for i = 1 to 5 do 

ipi •<— (1 — t) X 


-I- T X 


Ej=i “j ' ' ' 1 

return sample from 9 according to p distribution 


In [^, it is shown that, for a choice of constant 6’s, Ex- 
ponentiatedGradient can provide a regret bound on the 
optimal sequence of chosen 9 in the vector. In our case, the 
experimental results also show that ExponentiatedGradient 
is the best performing strategy. 

7. INCREMENTAL SOLUTION FOR OIM 

In our OIM framework, an IM algorithm is invoked once 
in every trial to select seeds. However, the state-of-the-art 
IM algorithms with good theoretical approximation bounds, 
such as CELF, TIM, and TIM-I-, are generally costly to run, 
especially for large graphs with high influence probabilities. 
For instance, in our experiments in the DBLP datasel|^ 
which has around 2,000,000 edges, the best known algorithm 
(TIM-h) also takes around half an hour to select the nodes 
for a trial. Since every run of OIM takes multiple trials, 
the running time can be too high in practical terms. To 
alleviate this issue, we explore in this section the possibility 
to increase the scalability of the OIM framework, by re-using 
computations between trials. 

The first observation is that all the IM algorithms with 
theoretical approximation bounds are sample based, and 
follow the general sampling process illustrated in Figure]^ a). 
Every time an algorithm requires a sample, it samples the 
influence graph based on the edge influence probabilities and 
stores it in a sample, say s. Moreover, their running time is 
dominated by the cost of sampling the influence graph (the 
thick arrow in Figure |^a)). For example, more than 99% of 
the computation of TIM-1- is spent in sampling the random 
reverse reachable sets in the influence graph [26| . 

Secondly, the size of the real-world feedback is relatively 
small compared with the number of edges in a graph. For 
instance, in DBLP with k = 1 and using TIM-f, the average 
IFnl is less than 1% of the total number of edges in the graph. 
This makes intuitive sense. Since samples are generated 
based on the influence graph, and the real-world feedback 
only influences a small part of the graph, it would only 
affect few samples taken from the updated influence graph 
in the next trial. This motivates us to explore methods 
which can save the computational effort, especially the effort 
in sampling, by reusing samples of previous trials, without 
incurring much error. 

7.1 Solution Framework 

To explain our approach, we introduce a sample manager 
(SM) which is responsible for the sampling procedure for the 

^Detailed description of the dataset is given in Section]^ 


sample-based IM algorithms. Generally speaking, when the 
IM algorithm requires a sample of the influence graph, it 
sends the request to SM, which will then return a sample to 
it. To enable an incremental approach that reuses the com¬ 
putational effort, SM stores the samples from the previous 
iterations in a sample pool. In the new trial, it attempts to 
reuse the stored samples, if possible, instead of sampling the 
influence graph again. 

The principle of SM is illustrated in Figure [^b). In a new 
trial, when the sample-based IM algorithm requires a sample, 
it sends requests to SM (Step 1). SM then randomly selects 
a sample s, which has not been used in this trial, from the 
sample pool (Step 2). After that, SM conducts two checks, 
called local check and global check, on s, whose purpose is 
to determine whether s is allowed to be reused after local 
and global updates performed in previous rounds (Step 3). 
If s passes these two checks, SM simply returns the sample 
to the IM algorithm (Step 4); otherwise, SM generates a 
new sample s' based on the current influence graph (Step 
5), and returns it to the IM algorithm (Step 6) as well as 
replaces s by s' in the sample pool (Step 7). 

In the above framework, assuming that conducting the 
local and global checks is much more efficient than sampling 
the influence graph and the ratio of reused samples is high, 
SM has the potential to significantly reduce the running time 
of the IM algorithm in the OIM framework. 

Next, we demonstrate how this principle can be applied in 
practice on the TIM- 1 - algorithm. Please note that the same 
principle can be easily applied to develop the incremental 
approaches for other sample-based IM algorithms. 

7.2 Case Study: TIM+ 

In this section, we demonstrate the case that TIM-|- is exe¬ 
cuted when an IM algorithm is called in OIM framework. For 
example, in Explore, TIM-|- is run with the input influence 
graph obtained by taking the mean of the random variable as 
the influence probability of the edge, i.e., pij = —. We 
next demonstrate how to develop the incremental approach 
for TIM-f in Exploit with SM. The principle also applies for 
Explore as well as CB. We focus on Exploit here and omit 
details for others. 

Briefly speaking, TIM- 1 - generates a set of random reverse 
reachable sets (or random RR sets) on the influence graph, 
and estimates the expected spread of nodes, based on the 
generated random RR sets. Here, an RR set for node v £ V, 
denoted by is a set of nodes which are: ( 1 ) generated on 
an instance of a randomly sampled influence graph g (an edge 
exists with a probability equal to its influence probability), 
and (2) able to reach v in the sampled graph g. In other 
words, yi £ Rv, there exists a path from itov in g. A random 
RR set is then an RR set where v is selected uniformly at 
random from V. We omit the formal definition of random RR 
sets as well as their generation and refer interested readers 
to for details. 

Let E[Ry) be the set of all incoming edges for nodes in 
i.e., E{Rv) = {(*, j)l(*ij) ^ E Aj £ Rv}. The next lemma is 
the foundation of the incremental approach for TIM- 1 -. 

Lemma 1. Given node v £ V , the occurrence probability of 
an RR set (Rv) keeps unchanged if the influence probabilities 
for edges in E{Rv) do not change. 

Proof. Let be a random variable for the existence of 
edge (i,j) £ E. We have (ij = 1 with probability of pij, and 
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Figure 3; Sample-based IM algorithms. 


= 0 with probability of 1 —Pij- Pr(7?„) is the probability 
that the following two events happen: (i) Vi € Rv, there 
exists a path from i to v, i.e., 3j € Rv s.t. = 1; and 
(ii) Wi ^ Rv, there exists no path from i to v, i.e., Wj € Rv, 
^ij = 0. And therefore, Pr(i?„) is some function of pij where 
{i,j) G E{Rv). Hence, \l Pij G E{Rv)) is unchanged, 

Pr(i?„) keeps unchanged, too. □ 

Let us consider SM introduced in Section [7d] The samples 
stored in SM for TIM-1- are the random RR sets described 
above. After each round, local and global graph updates may 
be performed according to the real world feedback. Suppose 
the current trial is n, for a randomly selected s (or Rv) from 
the sample pool, we have to conduct local and global checks 
for it. Lemmagives an intuition on how these checks can 
be performed for these checks. Pr(J?„) remains the same (or 
only deviates a bit) if the updates have no effect (or only 
some minor effects) on the influence probabilities for edges 
in E{Rv). 

Before we detail the local and global checks, let us first 
define the age of a sample s and the age of a node u. The 
age of s is the trial when s was sampled, and the age of a 
node u is the latest trial when the real world test attempted 
to activate u (regardless of the activation’s success). 

Local check. Let Rv's age to be t, and Eiocai be the 
set of edges that exist in the feedbacks from the t-th trial 
to the (n — l)-th trial, i.e., Eiocai = {{'i,j)\3q{t ^ q ^ n — 
1) s.t. {i,j) G Fq}. Local updates only affect edges that are 
included in the real world feedback, and so, Elocal<3E{Rv) = 0 
indicates that influence probabilities for edges in E(Rv) in 
the n-th trial are the same as the ones in the t-th trial. Hence, 
Rv is not affected by local updates. In other words, 

{Eiocai n E{Rv) = 0) => {Rv passes local check). 

We use the sample and node ages for an efficient local 
check as follows. 

Lemma 2 (Local Check). If for all u G Rv, u’s age 
is smaller than Rv ’s age, we have Eiocai H E{Rv) = 0. 

Proof. Recall that u’s age, denoted I, is the latest trial 
that the real world test tried to activate it. We have, 

{I < t) ^ {i{i,u) G E,{i,u,aiu) ^ Fq{t ^ q ^ n - 1)). 
Lemma is then a direct consequence by considering the 
definition of Eiocai ■ □ 

According to Lemma we store the sample as well as its 
age in the sample pool, and we also store the node’s age in 
a node activated history (refer to Figure |^c)). Then, the 
time complexity to do local check is 0(|i?„|) as the age’s 
information can be accessed in constant time. 

Global check. After global update is performed, the 
global a and /3 priors may be changed. Since they are shared 


by all edges, changes on global priors lead to changes on 
all edges’ influence probabilities. However, we observe that 
they will converge as we get more activation feedback from 
the real world. Intuitively, if the influence probabilities for 
edges in E{Rv) only deviate a bit, there is only minor effect 
on the random RR sets. Note that, only samples which 
pass local check will be then evaluated by the global check. 
And so, if the global priors when the sample s (or Rv) was 
generated are close to the current global priors, the influence 
probabilities for edges in E{Rv) do not change much. 

Let and /3^ be the priors at trial t, and the current priors 
are a and /3. We use a threshold r to measure whether two 
priors are close, moreover, whether global check is passed. 


f — 

\ a* -I- /3‘ 


a 

Q. + P 



^ {Rv passes global check) . 


Hence, in SM, we also store the priors when the sample was 
generated in the sample pool (Figure[^c)). And therefore, 
the global check is conducted in constant time 0 ( 1 ). 

Discussions. The total time complexity of conducting 
local and global checks on a sample Rv is 0(|i?„|). As 
mentioned in [^, the complexity of generating a sample Rv 
is of the order of the total in-degree for nodes in Rv, i.e., 
0(|£l(i?i,)|). Let d be the average in-degree for a node, we 
have \E{Rv)\ = d x ji?„| on average. This indicates that 
conducting checks for a sample is about d times faster than 
generating a new sample. Hence, the incremental approach 
for TIM- 1 - with SM can significantly save computation effort 
if the ratio of re-used samples is high. 

Note that if CB is employed, 6 may also be updated accord¬ 
ing to the real-world feedback. We design a similar mech¬ 
anism with global check, called 9 check, to verify whether 
Rv is allowed to be re-use. Let 6^ be the 6 when Rv was 
generated and a* be the standard deviation for global prior. 
We have 


(|0^(T* — 6a\ < t) => {Rv passes 9 check) . 

In the next section, we show our experimental results to 
verify our OIM framework. 


8. EXPERIMENTAL EVALUATION 

We now present the results. Section |8.1| describes the 
experiment settings. In Sections |8.2| and |8.3| we discuss our 
results on different datasets. 

8.1 Setup 

We developed a “real-world simulator” to mimic the user 
feedback process of Figure This simulator first uses a real 
social network to obtain a graph G. It then associates an 
influence probability to each edge in G, where Pij = 1/dj , 
with dj the in-degree of node j. This setting of influence 






























(a) Varying k under fixed budget (b) Varying k under fixed trials 
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Figure 4: Heuristic-based v.s. Explore-Exploit. 
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Table 2: Datasets 


Dataset 

NetHEPT 

NetPHY 

DBLP 

# of Nodes 

15K 

37K 

655K 

# of Edges 

59K 

231K 

2.1M 

avg. degree 

7.73 

12.46 

6.1 

max. degree 

341 

286 

588 


probability values is adopted in previous works §0[TT)[T^ 

MM- 

When the chosen seed nodes are tested on whether they can 
influence other nodes, the simulator runs a single independent 
cascade simulation on G, and obtains feedback information 
-F„, in a form of and An, the set of successfully 

activated nodes. We measure the effectiveness of an OIM 
solution by its influence spread in the real world, after N 
trials, as the total number of successfully activated nodes in 
these trials, i.e, | An\. We repeat each solution 10 times 
and report the average. 

Datasets. We have studied several real social network 
datasets. We have used NetHept and NetPhy are col¬ 
laboration networks, obtained from arXiv.org in the High 
Energy Physics Theory and Physics domains, respectively. 
We have also used the DBLP graph, which is an academic 
collaboration network. In these datasets, nodes represent au¬ 
thors, and edges representing co-authorship. These datasets 
are commonly used in the literature of influence maximiza¬ 
tion Table shows the details of these 

datasets. 

Options for OIM algorithm. We have evaluated sev¬ 
eral possible options for the seed selection and graph update 
components for our OIM solution: 

[Choosing seeds] 

• Heuristic-based strategies: Random, MaxDegree; 

• Explore-Exploit strategies: 1) Exploit contains only 
exploit algorithm; 2) e-greedy represents e-greedy al¬ 
gorithm; 3) CB is our Confidence-Bound explore-exploit 
algorithm with Exponentiated Gradient update. 

[Updating graph] 

• NO does not conduct any update; 

• LOG only local updates; 

• LSE local and global updates where Least Squares Esti¬ 
mation is adopted in global update; 

• MLE as LSE, but Maximum Likelihood Estimation is 
adopted. 

In our experiments, we compare the algorithms using com¬ 


binations of the above two components. Note that Random 
and MaxDegree do not rely on the influence probability of 
the edges, and they are not combined with update meth¬ 
ods. When a particular EE strategy is adopted, the update 
method would be specified, for instance, CB+MLE means that 
we use CB with MLE update. By default, we use MLE for up¬ 
dating the graph. Furthermore, if the EE strategy is used in 
choosing seeds, we use CB by default. 

When an IM algorithm is invoked in an EE strategy, we 
use TIM-b since it is the state-of-art influence maximization 
algorithm. We also compare the incremental approach with 
the non-incremental one for EE strategy. For example, we 
denote the incremental version for CB as CB-INC. 

Parameters. By default, the global prior is set to be 
B(l,19), 6 = { — 1,0,1} in CB, e = 0.1 in e-greedy, and 
T = 0.02 in the incremental approach. 

Our algorithms, implemented in C-|— 1 -, are conducted on a 
Linux machine with a 3.40 GHz Octo-Gore Intel(R) processor 
and 16GB of memory. Next, we focus on NetPHY, and 
evaluate different combinations of the algorithms in our OIM 
framework. We summarize our results for other datasets in 
Section 18.31 

8.2 Results on NetPHY 

Heuristic-based v.s. Explore—Exploit. We first fix 
the total budget and verify how the OIM algorithms perform 
with different number of trials. We set Budget = 50, and 
vary k in {1, 5,10, 25, 50}. By varying k, we essentially vary 
the total budget. For example, with fc = 5, 50 units of 
budget is invested over Y = 10 trials. Figure [4^ shows our 
results. Since Random only has influence spread less than 200 
on average, we do not plot it. We observe that the spread of 
MaxDegree does not change much since it does not depend 
on the real-world feedback. For CB, its spread increases when 
k decreases and it is better than MaxDegree when fc Si 10 (or 
Y > 5). Specifically, when fc = 1, CB is about 35% better 
than MaxDegree. The reason is that, for CB, a smaller fc 
indicates more chances to get real-world feedback, and thus, 
more chances to learn the real influence graph, which leads 
to a better result. Moreover, when fc = 50, all budget is 
invested once, which can be regarded as an offline solution, 
and produces the worst result for CB. This further indicates 
the effectiveness of our OIM framework. For CB-INC, it 
performs close to CB with only a small discount (around 5% 
for different fc) on the spread. It supports our claim that the 
incremental approach can perform without incurring much 
error. 
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Figure 5: Explore-exploit strategies 



(a) Different updates 



(b) Effect of priors 


Figure 6: Comparing different updating methods 


We next fix k and compare different algorithms in Figure [4b| 
The resuits are consistent with our previous findings that CB 
outperforms other variants. CB-INC produces similar results 
with CB. We observe that the gap between CB and MaxDegree 
increases with N and k. For example, at N = 50, CB is about 
20% better than MaxDegree when k = 5, and the percentage 
grows to 45% when fe = 25. The reason is that larger k 
and larger N give more chances for CB to learn the real 
influence graph. We also plot the result for TIM+ when the 
real influence probability is known, denoted as Real. This 
can be seen as an oracle, serving as a reference for other 
algorithms. We hnd that CB performs close to Real, and its 
discount on the spread decreases with N. For example, when 
k = 5, the discount decreases from 30% at = 10 to 13% at 
N = 50. This indicates that, with more real-world feedback, 
the learned graph for CB is closer to the real graph, and thus, 
leads to a closer result to Real. 

Explore—Exploit Strategies. We compare three ver¬ 
sions of the EE strategies for different k in Figure We 
observe that Exploit is the worst, since it may suffer from 
the wrong prediction of the influence probabilities and does 
not explore other potential high influencing nodes. CB is the 
best, especially, for small fe. When fe = 5, A1 = 50, CB is 
about 20% and 32% better than e-greedy and Exploit, re¬ 
spectively. The reason is that for a smaller fe, fewer feedback 
tuples are returned in one trial, which makes the learned 
influence graph converge to the real graph slower. Hence, the 
effect of exploration is strengthened, which is more favorable 
to CB. We have also conducted experiments for e-greedy by 
varying e. We observe that its performance is sensitive to e 
and £ = 0.1 is the best one in our results, but it is still worse 
than CB in all cases. 

Updating the uncertain influence graph. In Fig¬ 
ure we compare different updating methods for the un¬ 


certain influence graph. Although NO makes use of the prior 
knowledge about the influence graph to select seeds, it still 
performs worse than other update options. LOC is slightly 
better, but still worse than MLE and LSE, since it does not 
employ any global update and it suffers from the sparseness 
of the activations. MLE is the best (about 25% better than 
LSE and 40% better than LOC), which is consistent with the 
fact that MLE makes use of the full feedback to update the 
graph while LSE only utilizes the set of successfully activated 
nodes. 

We also test the updating methods with different priors 
(Figure [ 6 b| ) to check whether they are sensitive to the prior. 
We observe that while LOC and NO fluctuate a lot with different 
priors, MLE and LSE’s performance is very stable. In fact, 
during different runs of MLE and LSE with different priors, the 
global /3 values all converge to around 27. This supports the 
fact that the global updating techniques are crucial when we 
do not have good prior information. Even an inexact choice 
of prior will be generally fixed, minimizing the impact on 
performance. 

Efficiency. In Figure |7a| we illustrate the cumulative 
running time for running N trials for different algorithms. 
Random and MaxDegree are most efficient as they do not rely 
on any influence evaluation. With the help of incremental 
approach, CB-INC runs significantly faster than CB, and for 
the case where N ^ 10, it achieves about 10 times speedup. 
For instance, at N = 50, CB-INC reduces the running time 
by 88 %, compared to CB. This is intuitive, as in the first few 
trials the graph is more uncertain, and the updates affect the 
samples a lot. However, when N ^ 10, we observe that the 
global priors become more stable, leading to a high ratio of 
re-using samples (e.g., the ratio is about 80% to 99% when 
N ^ 10). Moreover, the average in-degree of NetPHY is 
12.46, making the time of generating a new sample about 
an order of magnitude slower than re-using a sample. These 
two factors together make CB-INC have a much more efficient 
performance than CB. 

We then show the efficiency results by fixing Budget = 50 
and varying fe in Figure [Tb] The running time of MaxDegree 
and Random is stable for various fe, while CB and CB-INC show 
a decline on efficiency when fe decreases. This is because a 
smaller fc indicates that more trials are required to invest all 
budget, and so, TIM-|- should be executed more often, for 
a general decrease in efficiency. Another observation is that 
the improvement of CB-INC over CB increases with fe. This 
further strengthens the utility of using CB-INC in practice. 
Figure [7b| and Figure [4^ together show a tradeoff of setting 
fe: a smaller fe leads to a better performance in spread but 
worse performance in efficiency. We suggest to set a small fe 
to ensure the algorithm’s better performance in spread. The 
value of fe will depend on how much total time that the user 
can afford. 

Effect of r. We also verify the effect of r in the in¬ 
cremental approach by varying r from 0.01 to 0.03 and 
fixing fe = 1, Budget = 50. We compare them with CB, the 
non-incremental algorithm. First, a smaller r gives better 
results in terms of influence spread. For instance, it leads 
to 3%, 5%, 15% discount in spread compared with CB for 
T = 0.01,0.02,0.03, respectively. However, a smaller r leads 
to a slowdown in efficiency since it has a stricter requirement 
in global check. For example, the running time for r = 0.01 
is about 28% slower than the one for r = 0.02 and 38% worse 
than the one for r = 0.03. 
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Figure 7: Cumulative running time 
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Figure 9: Efficiency on other datasets 




Figure 8: Effectiveness on other datasets 


Discussion. The OIM framework is highly effective in 
maximizing influence when the real influence probabilities 
are unknown. In this framework, MLE is the best updating 
method. Moreover, CB and CB-INC consistently outperform 
other algorithms. By using CB-INC, we can also significantly 
improve the efficiency of CB, with only a small discount in 
influence spread. 

8.3 Results for NetHEPT and DBLP 

Eigure and Figure show representative results for 
NetHEPT and DBLP. These results are consistent with 
the ones for NetPHY, where CB and CB-INC are close to the 
oracle (Real), and better than heuristic-based algorithms in 
maximizing influence spread. Eor efficiency, CB-INC signifi¬ 
cantly reduces the running time of CB, especially for a large 
dataset DBLP. For instance, at k = 1, N = 50, CB-INC saves 
16 hours compared with CB which costs 19 hours in total to 
get the result for DBLP. 


9. CONCLUSIONS 

In this paper, we examine how to perform influence maxi¬ 
mization when influence probabilities may not be known in 
advance. We develop a new solution, where IM is performed 
in multiple trials, and we have proposed explore-exploit 
strategies for this problem. We showed experimentally that 
explore-exploit based on the uncertainty in the graph per¬ 
forms well. We also proposed novel methods to update the 
knowledge of the graph based on the feedback received from 
the real world, and showed experimentally that they are 
effective in longer campaigns. Even when the influence prob¬ 
abilities are not known in advance, the influence spread of 
our solution is close to the spread using the real influence 
graph, especially when the number of trials increases. 


In the future, we will examine the scenario where budgets 
are different in each trial. We will extend our solution to 
handle other complex situations (e.g., the change of influence 


probability values over time), consider IM methods (e.g., 27 
0 ) that utilize community and topic information, and other 
influence propagation models, such as linear threshold or 
credit distribution 11|14|25 . Another direction is to increase 
the scalability of our methods; this may require distributed 
algorithm, such as distributed sampling. 
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